As the use of digital electronic communication networks has grown in recent years, the sophistication of internal and external network attacks in the form of viruses, Trojan horses, worms, and malware of various kinds has increased dramatically. Just as dramatic is the accelerated increase of network speeds and a corresponding drop in network costs, thereby driving the rapid adoption of networks. These and other factors have necessitated the development of innovative and more advanced network security measures.
For example, Intrusion Detection Systems (IDS) can often detect network attacks, but as passive systems they generally offer little more than after-the-fact notification. In contrast, Intrusion Prevention Systems (IPS) have been developed to complement traditional security products, such as firewalls, by proactively analyzing network traffic flows and active connections while scanning incoming and outgoing requests. As network traffic passes through the IPS, it is examined for malicious packets. Such examination may be performed by one or more “deep packet inspection engines” which perform “deep packet inspection” on some or all of the packets in the network traffic. Traffic is blocked if the IPS identifies it as posing a potential threat or as being associated with an unwanted application, while legitimate traffic is allowed to pass through the system unimpeded.
Properly implemented, an IPS can be an effective network security safeguard. There are, however, needs for improved IPS capabilities. For example, an IPS may include multiple deep packet inspection engines for performing deep packet inspection on traffic flows passing through the IPS because a single deep packet inspection engine, typically implemented as a microprocessor executing a suitable operating system and software, may not be capable of processing the flows at a sufficiently high throughput. Techniques for balancing network traffic load among multiple deep packet inspection engines in an IPS to increase the aggregate performance of such engines and thereby the overall performance of the IPS are disclosed in U.S. patent application Ser. No. 11/443,490, filed by Brian C. Smith, Alexander Sarin, and Hazem M. Kadaba on May 30, 2006, entitled “Intrusion Prevention System Edge Controller”; and U.S. patent application Ser. No. 11/782,840, filed by Gerald S. Stellenberg, Brian C. Smith, and James M. Rollette on Jul. 25, 2007, entitled “System and Method for Traffic Load Balancing to Manage Multiple Processors”.
Furthermore, the amount of time required to perform deep packet inspection on a single packet may vary widely from packet to packet. This amount of processing time, referred to as “inspection latency,” is affected, for example, by packet length and by the type of the packet. If the type of packet inspection applied to a particular type of packet requires that a complex regular expression (“regex”) pattern be matched against the packet, the inspection latency for that packet may be many orders of magnitude greater than the packet transmission speed. For example, the transmission time of a maximum-size Ethernet packet over a gigabit Ethernet link is 12.304 microseconds. Applying deep packet inspection to a packet using a recursive regex pattern may take 10 milliseconds or longer, i.e., approximately 1,000 times longer than the transmission speed.
Typically, however, up to 90-99% of network traffic does not require deep packet inspection. For such traffic, the inspection latency is much closer to the transmission latency than for packets requiring deep packet inspection. If, however, a particular network traffic flow queued for processing by a particular deep packet inspection engine includes a packet requiring deep packet inspection, the transmission of subsequent packets in the same flow will be delayed as such packets wait for the packet requiring deep packet inspection to be inspected.
Although employing multiple deep packet inspection engines within an IPS and applying load balancing techniques to spread packet flows among those engines, as described in the above-referenced patent applications, may provide some performance improvement, each deep packet inspection engine may still be responsible for processing thousands of active flows at a time. As a result, within a single deep packet inspection engine, the transmission of packets/flows not requiring deep packet inspection may still be delayed by the time required to inspect those packets/flows in the same engine which require deep packet inspection.
What is needed, therefore, are techniques for improving the performance of Intrusion Prevention Systems when processing flows having varying inspection requirements.